Some existing storage systems provide access to data storage through a REST interface via a coupling that presents a file system interface for managing, storing, updating and retrieving objects, data and metadata. There are a variety of systems that provide such an interface, these include the Content File System (CFS) from Caringo, File Connectors from Scality and Coronado Access Points from Bridgestor.
The file system side of these couplings uses traditional file system protocols such as NFS (Network File System), Server Message Block (SMB)/Common Internet File System (CIFS) and File Transfer Protocol (FTP) through file system drivers, or through other mechanisms that provide a file system interface to a client computer.
These existing storage systems are not optimal in that they must first either cache or stage data being read from or being written to the object storage; the caching or staging occurs via locally-accessible traditional block or NAS storage and is handled by the coupling server. On a client application read using a file system protocol, the coupling server must first retrieve a complete copy of the requested file or object data from the object storage; once a copy of the data has been successfully created on locally-accessible block storage, the coupling server then reads the data from locally-accessible storage and returns the requested data to the client application over the file system protocol. On open, read or write, the coupling server must first retrieve the file data from the object storage and then cache or stage the data on locally-accessible block or NAS storage. It can only then implement the original file system protocol requested action received from the client application by reading or updating the local cached or staged copy. After the client application has finished writing to the coupling server the coupling server must then transfer the locally-modified cached data to the object storage.
This requirement for caching or staging of data introduces latency and the risk of losing data if the coupling server itself or if the coupling server's locally-accessible cache or stage storage fails, or otherwise becomes unavailable before the data can be transferred to the object storage. Even storage systems that implement file or data manifests introduce similar latency and risk because the manifest may be unavailable corrupted or lost.
Therefore, improved systems and techniques are desired that address the above and other deficiencies in storage systems.